Exploiting User Behaviour in Prefetching WWW Documents

As the popularity of the World Wide Web increases, the amount of traffic results in major congestion problems for the retrieval of data over wide distances. To react to this, users and browser builders have implemented various prefetching and parallel retrieval mechanisms, which initiate retrieval of documents that may be required later. This additional traffic is even worsening the situation. Since we believe that this will remain the general approach for quite a while, we try to make use of the general technique but try to reduce the destructive effects by retrieving less content which remains finally unread. In our user-specific prefetch mechanism, the prefetching system gathers references by parsing the HTML pages the user browses, identifies the links to other pages, and puts the words describing the links into a keyword list. If such a word was already present in the list, its associated weight is incremented. Otherwise it is added to the table and a weighting factor allocated. We have designed and implemented a client based proxy-server with this mechanism. This paper shows the design and implementation of this prefetching proxy server, presents results and general considerations on this technique.